Electronic device for providing various functions through application using a camera and operating method thereof

ABSTRACT

Method and apparatuses are provided in which it is determined whether a document is included in an image provided from a camera of the electronic device. When the document is not included in the image, a first user interface is provided to a display of the electronic device that includes the image and at least one first graphic object for photographing the image. When the document is included in the image, a second user interface is provided to the display for providing a function related to the document. The second user interface includes the image, the at least one first graphic object, a second graphic object indicating that the document exists in the image, and at least one third graphic object for performing the function related to the document.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) to Korean Patent Application No. 10-2019-0018890, filed on Feb.19, 2019, in the Korean Intellectual Property Office, the disclosure ofwhich is incorporated herein by reference in its entirety.

BACKGROUND 1. Field

The disclosure relates to an electronic device which provides variousfunctions via an application using a camera, and an operation methodtherefor.

2. Description of Related Art

Electronic devices (e.g., a mobile terminal, a smart phone, a wearabledevice, and the like) are capable of providing various functions. Forexample, an electronic device may provide functions, such as, forexample, a voice communication function, a data communication function,a short-range wireless communication function (e.g., Bluetooth™, nearfield communication (NFC), or the like), a mobile communication function(e.g., 3^(rd) generation (3G), 4G, 5G, or the like), a music or videoreproduction function, a photographing or video recording function, anavigation function, or the like.

More particularly, an electronic device may provide a function ofrecognizing a document taken by a camera.

SUMMARY

In accordance with an aspect of the disclosure, an electronic device isprovided that includes a camera, a display, and a processor. Theprocessor is configured to determine whether a document is included inan image provided from the camera. When the document is not included inthe image, a first user interface is provided to the display thatincludes the image and at least one first graphic object forphotographing the image. When the document is included in the image, asecond user interface is provided to the display for providing afunction related to the document. The second user interface includes theimage, the at least one first graphic object, a second graphic objectindicating that the document exists in the image, and at least one thirdgraphic object for performing a function related to the document.

In accordance with an aspect of the disclosure, a method of anelectronic device is provided. It is determined whether a document isincluded in an image provided from a camera of the electronic device.When the document is not included in the image, a first user interfaceis provided to a display of the electronic device that includes theimage and at least one first graphic object for photographing the image.When the document is included in the image, a second user interface isprovided to the display for providing a function related to thedocument. The second user interface includes the image, the at least onefirst graphic object, a second graphic object indicating that thedocument exists in the image, and at least one third graphic object forperforming the function related to the document.

In accordance with an aspect of the disclosure, a computer-readablerecording medium is provided that stores a program to implementoperations, which include determining whether a document is included inan image provided from a camera. When the document is not included inthe image, a first user interface is provided to a display that includesthe image and at least one first graphic object for photographing theimage. When the document is included in the image, a second userinterface is provided to the display for providing a function related tothe document. The second user interface includes the image, the at leastone first graphic object, a second graphic object indicating that thedocument exists in the image, and at least one third graphic object forperforming the function related to the document.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of the disclosurewill be more apparent from the following detailed description, whentaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating an electronic device in a networkenvironment, according to an embodiment;

FIG. 2 is a flowchart illustrating a process of providing various userinterfaces based on whether a document is included in an image, by anelectronic device, according to an embodiment;

FIG. 3A is a diagram illustrating an example of a user interfaceprovided based on whether a document is included in an image, by anelectronic device, according to an embodiment;

FIG. 3B is a diagram illustrating an example of a user interfaceprovided based on whether a document is included in an image, by anelectronic device, according to an embodiment;

FIG. 4 is a diagram illustrating an example of a user interfaceincluding at least one graphic object related to a document, in anelectronic device, according to an embodiment;

FIG. 5 is a flowchart illustrating a process of performing a functioncorresponding to a user input when a document exists in an image, by anelectronic device, according to an embodiment;

FIG. 6A is a diagram illustrating an example of performing a functioncorresponding to a user input, by an electronic device, according to anembodiment;

FIG. 6B is a diagram illustrating an example of performing a functioncorresponding to a user input, by an electronic device, according to anembodiment;

FIG. 7 is a flowchart illustrating a process of providing various userinterfaces based on whether a document is recognizable, by an electronicdevice, according to an embodiment;

FIG. 8A is a diagram illustrating an example of a user interface thatprovides a document-related function based on the type of document, byan electronic device, according to an embodiment;

FIG. 8B is a diagram illustrating an example of a user interface thatprovides a document-related function based on the type of document, byan electronic device, according to an embodiment;

FIG. 8C is a diagram illustrating an example of a user interface thatprovides a document-related function based on the type of document, byan electronic device, according to an embodiment;

FIG. 8D is a diagram illustrating an example of a user interface thatprovides a document-related function based on the type of document, byan electronic device, according to an embodiment;

FIG. 8E is a diagram illustrating an example of a user interface thatprovides a document-related function based on the type of document, byan electronic device, according to an embodiment;

FIG. 9 is a flowchart illustrating a process of providing various userinterfaces based on a predetermined condition, by an electronic device,according to an embodiment;

FIG. 10A is a diagram illustrating an example of a user interfaceincluding an error report message related to the size of a document, inan electronic device, according to an embodiment;

FIG. 10B is a diagram illustrating an example of a user interfaceincluding an error report message related to the size of a document, inan electronic device, according to an embodiment;

FIG. 10C is a diagram illustrating an example of a user interfaceincluding an error report message related to the size of a document, inan electronic device, according to an embodiment;

FIG. 11A is a diagram illustrating an example of a user interfaceprovided based on the distortion level of a document in an electronicdevice, according to an embodiment;

FIG. 11B is a diagram illustrating an example of a user interfaceprovided based on the distortion level of a document in an electronicdevice, according to an embodiment; and

FIG. 11C is a diagram illustrating an example of a user interfaceprovided based on the distortion level of a document in an electronicdevice, according to an embodiment.

DETAILED DESCRIPTION

Embodiments are described in detail with reference to the accompanyingdrawings. The same or similar components may be designated by the sameor similar reference numerals although they are illustrated in differentdrawings. Detailed descriptions of constructions or processes known inthe art may be omitted to avoid obscuring the subject matter of thedisclosure. The embodiments and the terms used therein are not intendedto limit the technology disclosed herein to specific forms, and shouldbe understood to include various modifications, equivalents, and/oralternatives to the corresponding embodiments. A singular expression mayinclude a plural expression unless they are definitely different in acontext.

FIG. 1 is a block diagram illustrating an electronic device in a networkenvironment, according to an embodiment.

Referring to FIG. 1, an electronic device 101 in a network environment100 may communicate with an electronic device 102 via a first network198 (e.g., a short-range wireless communication network), or anelectronic device 104 or a server 108 via a second network 199 (e.g., along-range wireless communication network). According to an embodiment,the electronic device 101 may communicate with the electronic device 104via the server 108. The electronic device 101 includes a processor 120,memory 130, an input device 150, a sound output device 155, a displaydevice 160, an audio module 170, a sensor module 176, an interface 177,a haptic module 179, a camera module 180, a power management module 188,a battery 189, a communication module 190, a subscriber identificationmodule (SIM) 196, or an antenna module 197. In some embodiments, atleast one (e.g., the display device 160 or the camera module 180) of thecomponents may be omitted from the electronic device 101, or one or moreother components may be added in the electronic device 101. In someembodiments, some of the components may be implemented as singleintegrated circuitry. For example, the sensor module 176 (e.g., afingerprint sensor, an iris sensor, or an illuminance sensor) may beimplemented as embedded in the display device 160 (e.g., a display).

The processor 120 may execute, for example, software (e.g., a program140) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device 101 coupled with theprocessor 120, and may perform data processing or computation. Accordingto one embodiment, as at least part of the data processing orcomputation, the processor 120 may load a command or data received fromanother component (e.g., the sensor module 176 or the communicationmodule 190) in volatile memory 132, process the command or the datastored in the volatile memory 132, and store resulting data innon-volatile memory 134. The processor 120 may include a main processor121 (e.g., a central processing unit (CPU) or an application processor(AP)), and an auxiliary processor 123 (e.g., a graphics processing unit(GPU), an image signal processor (ISP), a sensor hub processor, or acommunication processor (CP)) that is operable independently from, or inconjunction with, the main processor 121. Additionally, oralternatively, the auxiliary processor 123 may be adapted to consumeless power than the main processor 121, or to be specific to a specifiedfunction. The auxiliary processor 123 may be implemented as separatefrom, or as part of the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one component (e.g., the display device 160,the sensor module 176, or the communication module 190) among thecomponents of the electronic device 101, instead of the main processor121 while the main processor 121 is in an inactive (e.g., sleep) state,or together with the main processor 121 while the main processor 121 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 123 (e.g., an ISP or a CP) may beimplemented as part of another component (e.g., the camera module 180 orthe communication module 190) functionally related to the auxiliaryprocessor 123.

The memory 130 may store data used by at least one component (e.g., theprocessor 120 or the sensor module 176) of the electronic device 101.The data may include, for example, software (e.g., the program 140) andinput data or output data for a command related thereto. The memory 130includes the volatile memory 132 or the non-volatile memory 134.

The program 140 may be stored in the memory 130 as software, andincludes an operating system (OS) 142, middleware 144, or an application146.

The input device 150 may receive a command or data to be used by anothercomponent (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. The inputdevice 150 may include, for example, a microphone, a mouse, a keyboard,or a digital pen (e.g., a stylus pen).

The sound output device 155 may output sound signals to the outside ofthe electronic device 101. The sound output device 155 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record, and the receivermay be used for an incoming calls. According to an embodiment, thereceiver may be implemented as separate from, or as part of the speaker.

The display device 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. The display device 160 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaydevice 160 may include touch circuitry adapted to detect a touch, orsensor circuitry (e.g., a pressure sensor) adapted to measure theintensity of force incurred by the touch.

The audio module 170 may convert a sound into an electrical signal andvice versa. According to an embodiment, the audio module 170 may obtainthe sound via the input device 150, or output the sound via the soundoutput device 155 or a headphone of an external electronic device (e.g.,an electronic device 102) directly (e.g., wiredly) or wirelessly coupledwith the electronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 176 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., wiredly) orwirelessly. According to an embodiment, the interface 177 may include,for example, a high definition multimedia interface (HDMI), a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected with the externalelectronic device (e.g., the electronic device 102). According to anembodiment, the connecting terminal 178 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 179 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 180 may capture a still image or moving images.According to an embodiment, the camera module 180 may include one ormore lenses, image sensors, ISPs, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. According to one embodiment, the power managementmodule 188 may be implemented as at least part of, for example, a powermanagement integrated circuit (PMIC).

The battery 189 may supply power to at least one component of theelectronic device 101. According to an embodiment, the battery 189 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device 104, or the server 108) andperforming communication via the established communication channel. Thecommunication module 190 may include one or more communicationprocessors that are operable independently from the processor 120 (e.g.,the AP) and supports a direct (e.g., wired) communication or a wirelesscommunication. According to an embodiment, the communication module 190may include a wireless communication module 192 (e.g., a cellularcommunication module, a short-range wireless communication module, or aglobal navigation satellite system (GNSS) communication module) or awired communication module 194 (e.g., a local area network (LAN)communication module or a power line communication (PLC) module). Acorresponding one of these communication modules may communicate withthe external electronic device via the first network 198 (e.g., ashort-range communication network, such as Bluetooth™, wireless-fidelity(Wi-Fi) direct, or infrared data association (IrDA)) or the secondnetwork 199 (e.g., a long-range communication network, such as acellular network, the Internet, or a computer network (e.g., LAN or widearea network (WAN)). These various types of communication modules may beimplemented as a single component (e.g., a single chip), or may beimplemented as multi components (e.g., multi chips) separate from eachother. The wireless communication module 192 may identify andauthenticate the electronic device 101 in a communication network, suchas the first network 198 or the second network 199, using subscriberinformation (e.g., international mobile subscriber identity (IMSI))stored in the subscriber identification module 196.

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. According to an embodiment, the antenna module197 may include an antenna including a radiating element composed of aconductive material or a conductive pattern formed in or on a substrate(e.g., PCB). The antenna module 197 may include a plurality of antennas.In such a case, at least one antenna appropriate for a communicationscheme used in the communication network, such as the first network 198or the second network 199, may be selected, for example, by thecommunication module 190 (e.g., the wireless communication module 192)from the plurality of antennas. The signal or the power may then betransmitted or received between the communication module 190 and theexternal electronic device via the selected at least one antenna.Another component (e.g., a radio frequency integrated circuit (RFIC))other than the radiating element may be additionally formed as part ofthe antenna module 197.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 104 via the server 108 coupled with the second network 199. Eachof the electronic devices 102 and 104 may be a device of a same type as,or a different type, from the electronic device 101. All or some ofoperations to be executed at the electronic device 101 may be executedat one or more of the external electronic devices 102, 104, or 108. Forexample, if the electronic device 101 should perform a function or aservice automatically, or in response to a request from a user oranother device, the electronic device 101, instead of, or in additionto, executing the function or the service, may request the one or moreexternal electronic devices to perform at least part of the function orthe service. The one or more external electronic devices receiving therequest may perform the at least part of the function or the servicerequested, or an additional function or an additional service related tothe request, and transfer an outcome of the performing to the electronicdevice 101. The electronic device 101 may provide the outcome, with orwithout further processing of the outcome, as at least part of a replyto the request. To that end, a cloud computing, distributed computing,or client-server computing technology may be used, for example.

According to an embodiment, when running a first application, theprocessor 120 may control the camera module 180 so as to obtain at leastone image, and may control the display device 160 so as to display atleast one image on a preview screen. The first application may be anapplication that provides a photography function using a camera. Forexample, the first application may be a camera application that providesat least one of a photography function or a video recording function.However, embodiments described below may not be limited thereto, and maybe applied to various applications using a camera.

According to an embodiment, the processor 120 may determine whether adocument is included in at least one image obtained by the camera module180. The processor 120 may determine whether a document is included inan image based on whether text exists in at least one image, and/orwhether a code provided in a predetermined form (e.g., a bar code or aQR code) is included in at least one image. For example, if text, suchas characters and/or numbers, or a code provided in the predeterminedform, is not included in the image, the processor 120 may determine thata document is not included in an image. As another example, if text,such as characters and/or numbers, exists in an image, the processor 120may determine whether a document is included in the image based on theshape of an object including the text in the image. For example, theprocessor 120 may determine whether a document is included in the imagebased on the number of segments (or sides) or the number of vertices ofan object including text in the image. If the number of segments and/orthe number of vertices detected from around the text do not satisfy apredetermined condition, the processor 120 may determine that a documentis not included in the image. If the number of segments and/or thenumber of vertices detected from around the text satisfy a predeterminedcondition, the processor 120 may determine that a document is includedin the image. When four vertices are detected from around the text, andtext exists inside a quadrangular shape that connects the four vertices,the processor 120 may determine that a document is included in theimage. As another example, if a code in a predetermined form (e.g., abar code or a QR code) is included in the image, the processor 120 maydetermine that a document is included in the image. The above-describedmethods that determine whether text is included in the image are merelyexamples, and the embodiments of the disclosure are not limited thereto.

According to an embodiment, the processor 120 may provide a userinterface based on whether a document is included in an image. If adocument is not included in at least one image, the processor 120 mayprovide a first user interface for photographing an image. The firstuser interface may include at least one first graphic object related tophotographing an image. A graphic object may include, for example, atleast one of a button, an icon, a symbol, an item, or a menu, which isvisually displayed. For example, the first user interface may include atleast one of a first button for providing a photography function or asecond button for providing a video recording function. If a document isincluded in at least one image, the processor 120 may provide a seconduser interface for providing a document-related function. The seconduser interface may include at least one first graphic object related tophotographing an image, and may additionally include at least one of asecond graphic object indicating that a document exists in an image andat least one third graphic object that provides a document-relatedfunction. The second graphic object indicating that a document exists inan image may include at least one of an icon, a symbol, an item, or agraphic effect, which visually reports that the existence of a documentis recognized. For example, the second graphic object indicating that adocument exists in an image may include at least one of a predeterminedsymbol or a graphic effect that indicates an area where the existence ofa document is recognized (or identified). The at least one third graphicobject providing a document-related function may include, for example,at least one of a scan button for providing a function of scanning adocument, a payment button for providing a payment function based oninformation included in a document, a web access button for providing afunction of accessing a web page based on information included in adocument, a schedule add button for providing a function of adding aschedule based on information included in a document, a contactinformation add button for providing a function of adding contactinformation based on information included in a document, or anadditional information search button for providing a function ofsearching for additional information based on information included in adocument. The above-mentioned third graphic objects are merely examples,and embodiments are not limited thereto. The second user interface mayinclude, for example, at least one of a first button for providing aphotography function or a second button for providing a video recordingfunction, and may additionally include a symbol indicating the existenceof a document in an image, or at least one of a scan button, a webaccess button, a payment button, or a schedule add button. A thirdgraphic object to be included in the second user interface may bedetermined based on the document type of a document included in animage, and/or information included in the document.

According to an embodiment, the processor 120 may perform aphotography-related function or may perform a document-related functionbased on a user input in a state in which the second user interface isdisplayed. In the state in which the second user interface is displayed,if a user input to at least one first graphic object is detected, theprocessor 120 may perform a photography function or a video recordingfunction. An image obtained via the photographic function or the videorecording function may include an image displayed on a preview screen.In the state in which the second user interface is displayed, if a userinput to at least one third graphic object is detected, the processor120 may perform a document-related function. For example, if the thirdgraphic object on which a user input is detected is an object forproviding a scan function, the processor 120 may extract and store adocument from an image. As another example, if the third graphic objecton which a user input is detected is an object for providing a web pageaccess function, the processor 120 may obtain web address informationfrom a document included in an image, and may access a corresponding webpage using the obtained web address. As another example, if the thirdgraphic object on which a user input is detected is an object thatprovides a payment function, the processor 120 may obtain paymentinformation from a document included in an image, and may perform apayment function using the obtained payment information. As anotherexample, if the third graphic object on which a user input is detectedprovides a schedule add function, the processor 120 may obtain scheduleinformation from a document included in an image, and may add theobtained schedule information to a schedule management application.

According to an embodiment, the processor 120 may determine whether adocument included in an image is recognizable, and may provide a userinterface based on the recognizability. According to an embodiment, theprocessor 120 may determine whether the document included in the imageis recognizable, based on at least one of the size of the document orthe distortion level of the document. For example, if the size of thedocument is greater than a predetermined maximum size, or less than apredetermined minimum size, the processor 120 may determine that thedocument included in the image is not recognizable. As another example,if the distortion level of the document is beyond a predetermined levelrange, the processor 120 may determine that the document included in theimage is not recognizable. If the document included in the image isrecognizable, the processor 120 may provide a second user interface. Ifthe document included in the image is not recognizable, the processor120 may provide a third user interface including an error reportmessage, which indicates that the document is not recognizable. Theerror report message may include, for example, a message that requestsadjustment of at least one of a photography location, a photographingdirection, or a subject distance for photography (e.g., the distancebetween the document and the electronic device). The third userinterface may include at least one first graphic object related tophotographing and an error report message, and may additionally includeat least one of a second graphic object indicating that a documentexists in an image or at least one third graphic object that provides adocument-related function. The at least one third graphic objectincluded in the third user interface may be in a deactivated state. Forexample, the at least one third graphic object included in the thirduser interface may be deactivated so as to indicate that adocument-related function is not allowed to perform.

According to an embodiment, the processor 120 may determine at least onedocument-related function based on at least one of the document type ofthe document included in the image or information included in thedocument. The processor 120 may determine at least one third graphicobject to be included in the second user interface based on the at leastone determined function. The processor 120 may identify the documenttype of the document included in the image, and may determine whetherthe identified type of document is a type providing a single function ora type providing a plurality of functions based on a table listing arelated function for each type of document. The processor 120 maydetermine the type of document based on information included in thedocument or the form of the document.

For example, if a table is detected from the document included in theimage, or if schedule information including time and/or placeinformation is detected, the processor 120 may determine the type ofdocument as a first type (e.g., a poster, a memo, or a calendar). Asanother example, if a code in a predetermined form is detected, theprocessor 120 may determine the type of document as a second type (e.g.,a barcode or a QR code). As another example, if at least one ofpublisher information or author information is detected from thedocument included in the image, the processor 120 may determine the typeof document as a third type (e.g., a book). As another example, if atleast one piece of information among a user name, a company name, acompany address, a phone number, a fax number, or an email address isdetected from the document included in the image, the processor 120 maydetermine the type of document as a fourth type (e.g., a business card).As another example, if information corresponding to the first type tothe fourth type is not detected, the processor 120 may determine thetype of document as a fifth type (e.g., miscellaneous documents). Themethod of determining the type of document is merely an example, andembodiments are not limited thereto.

The table listing a related function for each type of document may be,for example, a table listing a related function for each type, such as“a document type: a first type (e.g., a poster, a memo, or a calendar),a related function: a scan function or a schedule add function”, “adocument type: a second type (e.g., a QR code or a barcode), a relatedfunction: a payment function or a web access function”, a document type:a third type (e.g., a book), a related function: a scan function or anadditional information search function”, “a document type: a fourth type(e.g., a business card), a related function: a scan function or, acontact information add function”, and “a document type: a fifth type(e.g., miscellaneous documents), a related function: a scan function”.Embodiments of the present disclosure are not limited to theabove-described examples.

If a single related function is determined for the identified type ofdocument, the processor 120 may determine the type of document as a typethat provides a single function, and may determine a third graphicobject that is capable of performing the single function. If a pluralityof related functions are determined for the identified type of document,the processor 120 may determine the type of document as a type thatprovides a plurality of functions, and may determine a third graphicobject capable of performing a single function or a plurality of thirdgraphic objects corresponding to the plurality of functions on the basisof information included in the document.

For example, if the document included in the image is the first type,the processor 120 may recognize text included in the document, and maydetermine whether schedule information (e.g., time information or placeinformation) is included in the document. If schedule information isincluded in the document, the processor 120 may determine a scanfunction and a schedule add function as document-related functions, andmay determine a scan button for providing a function of scanning adocument and a schedule add button for providing a function of addingschedule information to a predetermined application (e.g., a calendarapplication or a schedule application) as third graphic objects. Ifschedule information is not included in the document, the processor 120may determine a scan function as a document-related function, and maydetermine a scan button for providing a function of scanning a documentas a third graphic object.

As another example, if the document included in the image is the secondtype (e.g., a QR code or a barcode), the processor 120 may analyze acode in the document, and may determine whether payment information andweb address information are included. If the code in the documentincludes only web address information, the processor 120 may determine aweb access button for accessing a corresponding web page and providinginformation, as a third graphic object. If the code in the documentincludes web address information and payment information, the processor120 may determine a web access button and a payment button as thirdgraphic objects.

As another example, if the document included in the image is the thirdtype (e.g., a book, a research paper, or a leaflet), the processor 120may recognize text included in the document, and may determine whethersearchable information is included in the document. If searchableinformation is included in the document, the processor 120 may determinea scan function and an additional information search function asdocument-related functions, and may determine a scan button forproviding a scan function and a search button for providing anadditional information search function as third graphic objects.

As another example, if the document included in the image is the fourthtype (e.g., a business card), the processor 120 may determine a scanfunction and a contact information add function as document-relatedfunctions, and may determine a scan button for providing a scan functionand a contact information add button for providing a function ofregistering contact information included in a business card with anelectronic device, as third graphic objects.

As another example, if the document included in the image is the fifthtype (e.g., miscellaneous documents), the processor 120 may determine ascan function as a document-related function, and may determine a scanbutton that provides a scan function as a third graphic object. Theprocessor 120 may provide the second user interface including the atleast one determined third graphic object.

In the above description, the processor 120 may determine at least oneof whether a document is included in an image, the document type of adocument included in an image, or a document-related function, based ona document-related database (DB) included in the memory 130 of theelectronic device 101. The document-related DB may include various typesof information (e.g., a reference value, a condition, a rule fordetermining a document, or a function), which are needed to determine atleast one of whether a document is included in an image, the documenttype of a document included in an image, and a document-relatedfunction. If the processor 120 fails to determine at least one ofwhether a document is included in an image, the document type of adocument included in an image, and a document-related function, usingthe document-related DB included in the memory 130, the processor 120may transmit the image to an external device (e.g., the electronicdevice 102, the electronic device 104, or the server 108), and mayreceive at least one piece of information among whether a document isincluded in the image, the document type of a document included in theimage, or a document-related function from the external device (e.g.,the electronic device 102, the electronic device 104, or the server108). The document-related DB included in the memory 130 may be updatedvia interoperation with the external device (e.g., the electronic device102, the electronic device 104, or the server 108), and may be updatedby learning.

According to an embodiment, an electronic device may include a camera, adisplay, and a processor. The processor is configured to determinewhether a document is included in an image provided from the camera.When a document is not included in the image, the processor isconfigured to provide a first user interface including at least onefirst graphic object for photographing the image. When a document isincluded in the image, the processor is configured to provide a seconduser interface for providing a function related to the document. Thesecond user interface includes the at least one first graphic object, asecond graphic object indicating that a document exists in the image,and at least one third graphic object for performing a function relatedto a document.

The second graphic object may include at least one of a predeterminedsymbol indicating that a document exists in the image or a graphiceffect indicating an area where a document is recognized in the image.

When a document is included in the image, the processor is configured todetermine whether the document is recognizable, based on whether thedocument satisfies a predetermined condition, and when the document isrecognizable, the processor is configured to determine at least onefunction related to the document based on a type of document orinformation included in the document.

The at least one function related to the document may include at leastone of a function of scanning the document, a function of searching foradditional information related to the document, a payment function basedon information included in the document, a web access function based oninformation included in the document, a schedule add function based oninformation included in the document, and a function of adding contactinformation based on information included in the document.

The second user interface may include a third graphic object thatprovides a function of scanning the document. When a user input to thethird graphic object, which provides the function of scanning thedocument, is detected in the state in which the second user interface isprovided, the processor is configured to extract and store only thedocument from the image. When a user input to the first graphic objectis detected in the state in which the second user interface is provided,the processor is configured to photograph the image including thedocument and store the image.

The processor may determine whether at least one of a size of thedocument or a distortion level of the document satisfies thepredetermined condition.

When the document is not recognizable, the processor may provide a thirduser interface including an error report message related to at least oneof the size of the document or the distortion level of the document, andthe third user interface may further include at least one of the atleast one first graphic object or the second graphic object indicatingthat a document exists in the image.

The third user interface may further include at least one third graphicobject for performing a function related to the document, and the atleast one third graphic object may be in a deactivated state.

FIG. 2 is a flowchart 200 illustrating a process of providing varioususer interfaces based on whether a document is included in an image, byan electronic device, according to an embodiment. In the embodimentprovided hereinafter, operations may be performed sequentially, but itis not necessary. For example, the order of operations may be changed,and at least two operations may be performed in parallel. Here, anelectronic device may be the electronic device 101 of FIG. 1.Hereinafter, at least some steps of FIG. 2 will be described withreference to FIGS. 3A, 3B, and 4.

FIG. 3A is a diagram illustrating an example of a user interfaceprovided based on whether a document is included in an image, by anelectronic device, according to an embodiment. FIG. 3B is a diagramillustrating an example of a user interface provided based on whether adocument is included in an image, by an electronic device, according toan embodiment. FIG. 4 is a diagram illustrating an example of a userinterface including at least one graphic object related to a document,in an electronic device, according to an embodiment.

Referring to flowchart 200 of FIG. 2, an electronic device (e.g., theprocessor 120 of FIG. 1) determines whether a document is included in animage provided from a camera 180, in step201. The processor 120 maydetermine whether a document is included in at least one image providedfrom the camera 180 based on whether text exists in at least one imageand/or a code in a predetermined form (e.g., a barcode or a QR code) isincluded. For example, if text or a code provided in the predeterminedform is not included in an image displayed on a preview screen, theprocessor 120 determines that a document is not included in the image.As another example, if text exists in the image, the processor 120detects a segment or a vertex of an object including the text in theimage, and if the number of the detected segments and/or the number ofvertices does not satisfy a predetermined condition, the processor 120determines that a document is not included in the image. If the numberof segments and/or the number of vertices detected from around the textsatisfies a predetermined condition, the processor 120 may determinethat a document is included in the image. As another example, if a codein a predetermined form is included in the image, the processor 120determines that a document is included in the image. The above-describedmethods of determining whether text is included in an image are merelyexamples, and embodiments of the disclosure are not limited thereto.

If a document is not included in the image, the electronic device (e.g.,the processor 120) provides a first user interface for photographing animage, in step203. The first user interface may include at least onefirst graphic object related to photographing an image. A graphic objectmay include, for example, at least one of a button, an icon, a symbol,an item, or a menu, which is visually displayed. For example, asillustrated in FIG. 3A, if a document is not included in the imagedisplayed on a preview screen, the processor 120 provides a first userinterface including a photographing button 301 for providing aphotography function and a video recording button 303 for providing avideo recording function.

If a document is included in the image, the electronic device (e.g., theprocessor 120) provides a second user interface for providing adocument-related function, in step 205. The second user interface mayinclude at least one first graphic object related to photographing animage, and may additionally include at least one of a second graphicobject indicating that a document exists in an image or at least onethird graphic object that provides a document-related function. Thesecond graphic object indicating that a document exists in an image mayinclude at least one of an icon, a symbol, an item, or a graphic effectwhich visually reports that the existence of the document is recognized.For example, the second graphic object indicating that a document existsin an image may include at least one of a predetermined symbol or agraphic effect that indicates an area where a document is recognized.The at least one third graphic object that provides a document-relatedfunction may include, for example, at least one of a scan button forproviding a function of scanning a document, a payment button forproviding a payment function based on information included in adocument, a web access button for providing a function of accessing aweb page based on information included in a document, or an add buttonfor providing a function of adding a schedule based on informationincluded in a document. The above-mentioned third graphic objects aremerely examples, and embodiments of the present disclosure are notlimited thereto.

For example, as illustrated in FIG. 3B, if a document is included in theimage displayed on a preview screen, the processor 120 provides a seconduser interface including a photographing button 301 that provides aphotography function, a video recording button 303 for providing a videorecording function, a scan button 313 for providing a function ofscanning a document, a symbol 317 indicating that a document exists inan image, and a graphic effect 315 indicating an area where a documentis recognized. The display location and/or the size of a second graphicobject and/or a third graphic object to be included in the second userinterface may be set and changed by a designer and/or a user.

For example, as illustrated in FIG. 4, a scan button 401, 403, 405, and407, which is a third graphic object and a symbol 411, 413, 415, and417, which is a second graphic object and indicates the existence of adocument may be displayed in an area predetermined by a designer and/ora user. The symbol that is a second graphic object and indicates theexistence of a document may not trigger a predetermined function, andmay be displayed on a photographing button as shown in diagram 413 ofFIG. 4. As another example, the processor 120 may dynamically change thedisplay location of a second graphic object and/or a third graphicobject based on the location of a document included in an image. Forexample, every time that the location of a document in an image ischanged, the processor 120 may dynamically change the display locationof a third graphic object based on the changed location of the document.A third graphic object included in the second user interface may bedetermined based on the type of a document included in an image, and/orinformation included in the document.

FIG. 5 is a flowchart 500 illustrating a process of performing afunction corresponding to a user input when a document exist in animage, by an electronic device, according to an embodiment. Operationsof FIG. 5, which are described below, may be operations subsequent tostep 205 of FIG. 2. In the embodiment provided hereinafter, operationsmay be performed sequentially, but it is not necessary. For example, theorder of operations may be changed, and at least two steps may beperformed in parallel. Here, an electronic device may be the electronicdevice 101 of FIG. 1. Hereinafter, at least some operations of FIG. 5will be described with reference to FIGS. 6A and 6B.

FIG. 6A is a diagram illustrating an example of performing a functioncorresponding to a user input, by an electronic device, according to anembodiment. FIG. 6B is a diagram illustrating an example of performing afunction corresponding to a user input, by an electronic device,according to an embodiment.

Referring to FIG. 5, an electronic device (e.g., the processor 120 ofFIG. 1) detects a user input to a second user interface, in step 501.The second user interface may be the same as the second user interfacedescribed with reference to step 205 of FIG. 2. A user input may includeat least one of a single touch, a double touch, a long touch, or apressure touch.

The electronic device (e.g., the processor 120) determines whether auser input is an input with respect to a third graphic object, in step503. For example, the processor 120 may determine whether a user inputto a third graphic object is detected, or whether a user input to afirst graphic object is detected, among the graphic objects included inthe second user interface.

If the user input is an input to the third graphic object, theelectronic device (e.g., the processor 120) performs a document-relatedfunction mapped to the third graphic object, in step 505. As illustratedin FIG. 2, the third graphic object may include, for example, at leastone of a scan button, a payment button, a web access button, or aschedule add button. For example, as illustrated in FIG. 6A, if a userinput 601 to a scan button is detected, the processor 120 extracts adocument from an image, and performs a function of storing the extracteddocument 603. As another example, if a user input to a payment button isdetected, the processor 120 may obtain payment information from adocument included in an image, and may perform a payment function basedon the payment information. If a user input to a web access button isdetected, the processor 120 may obtain web address information from adocument included in an image, and may perform a function of accessing acorresponding web page based on the web address information. If a userinput to a schedule add button is detected, the processor 120 may obtainschedule information from a document included in an image, and may addthe obtained schedule information to an application that manages auser's schedules (e.g., a calendar application or a schedule managementapplication).

If a user input is not an input to the third graphic object (if the userinput is an input to the first graphic object), the electronic device(e.g., the processor 120) performs a photography-related function mappedto the first graphic object where the user input is detected, in step507. The first graphic object may include, for example, at least one ofa photographing button or a video recording button, as illustrated inFIG. 1. For example, as illustrated in FIG. 6B, if a user input 611 to aphotographing button is detected, the processor 120 performs a functionof obtaining and storing an image which is provided from a camera 180,and is displayed on a preview screen, as an image obtained byphotography. This may be the same as a normal photography function.

FIG. 7 is a flowchart 700 illustrating a process of providing varioususer interfaces based on whether a document is recognizable, by anelectronic device, according to an embodiment. Operations of FIG. 7described below may be the detailed operations of at least some of steps201, 203, and 205 of FIG. 2. In the embodiment provided hereinafter,operations may be performed sequentially, but it is not necessary. Forexample, the order of operations may be changed, and at least two stepsmay be performed in parallel. Here, an electronic device may be theelectronic device 101 of FIG. 1. Hereinafter, at least some operationsof FIG. 7 will be described with reference to FIGS. 8A to 8E.

FIG. 8A is a diagram illustrating an example of a user interface thatprovides a document-related function based on the type of document, byan electronic device, according to an embodiment. FIG. 8B is a diagramillustrating an example of a user interface that provides adocument-related function based on the type of document, by anelectronic device, according to an embodiment. FIG. 8C is a diagramillustrating an example of a user interface that provides adocument-related function based on the type of document, by anelectronic device, according to an embodiment. FIG. 8D is a diagramillustrating an example of a user interface that provides adocument-related function based on the type of document, by anelectronic device, according to an embodiment. FIG. 8E is a diagramillustrating an example of a user interface that provides a documentrelated function based on the type of document, by an electronic device,according to an embodiment.

Referring to flowchart 700 of FIG. 7, an electronic device (e.g., theprocessor 120 of FIG. 1) may operates a camera (e.g., the camera module180 of FIG. 1), and obtains at least one image, in step 701. Whenrunning a first application, the processor 120 operates the camera 180,and may obtain at least one image from the camera. The processor 120 maycontrol the display device 160 so as to display the at least one imageobtained from the camera module 180 on a preview screen. The firstapplication may be an application that provides a photography functionusing a camera. For example, the first application may be a cameraapplication that provides at least one of a photography function or avideo recording function.

The electronic device (e.g., the processor 120) determines whether adocument is included in an image, in step 703. The step of determiningwhether a document is included in the image may be at least partiallythe same as step 201 of FIG. 2. The processor 120 may determine whethera document is included in the image based on a document-related DBincluded in a memory 130 of the electronic device 101. Thedocument-related DB may include various types of information (e.g., areference value, a condition, a rule of determining a document, or afunction) needed to determine whether a document is included in theimage. If it is identified that determining whether a document isincluded in the image is difficult using the document-related DBincluded in the memory 130, the processor 120 may transmit the image toan external device (e.g., the electronic device 102, the electronicdevice 104, or the server 108) and may request the external device todetermine whether a document is included in the image. In response tothe request, the processor 120 may receive a response signal includinginformation indicating whether a document is included in the image, fromthe external device (e.g., the electronic device 102, the electronicdevice 104, or the server 108). The response signal may additionallyinclude at least one information associated with the document type of adocument included in the image or a document-related function, inaddition to the information indicating whether a document is included inthe image.

If it is determined that a document is not included in the image, theelectronic device (e.g., the processor 120) performs step 203 of FIG. 2.

If it is determined that a document is included in the image, theelectronic device (e.g., the processor 120) determines whether thedocument included in the image is recognizable, in step 705. Accordingto an embodiment, the processor 120 may determine whether the documentincluded in the image is recognizable, based on at least one of the sizeof the document or the distortion level of the document. For example, ifthe size of the document is greater than a predetermined maximum size,or less than a predetermined minimum size, the processor 120 maydetermine that the document included in the image is not recognizable.As another example, if the distortion level of the document is beyond apredetermined level range, the processor 120 may determine that thedocument included in the image is not recognizable.

If the document included in the image is recognizable, the electronicdevice (e.g., the processor 120) determines whether the document type ofthe document is a type providing a plurality of functions in step 707.The processor 120 may identify the document type of the documentincluded in the image, and may determine whether a singledocument-related function or a plurality of document-related functionsare used based on the type of document. For example, based on theinformation included in the document or the form of the document, theprocessor 120 may determine the type of document, and may determinewhether a single document-related function or a plurality ofdocument-related functions are used based on a table listing a relatedfunction for each type of document. The table listing a related functionfor each type of document may be, for example, a table listing a relatedfunction for each type, such as “a document type: a first type, arelated function: a scan function or a schedule add function”, “adocument type: a second type, a related function: a payment function ora web access function”, “a document type: a third type, a relatedfunction: a scan function or an additional information search function”,“a document type: a fourth type, a related function: a scan function, acontact information add function”, and “a document type: a fifth type, arelated function: a scan function”. If the identified type of documentis any one of the first type to the fourth type, the processor 120 maydetermine the type of document as a type providing a plurality offunctions. If the identified type of document is the fifth type, theprocessor 120 may determine the type of document as a type that isincapable of providing a plurality of functions (a type providing asingle function).

If the type of document is a type providing a plurality of functions,the electronic device (e.g., the processor 120) determines a single or aplurality of functions as document-related functions based oninformation included in the document, in step 709 and provides a seconduser interface including at least one third graphic object forperforming the document-related function, in step 711. For example, ifthe document included in the image is the first type, the processor 120recognizes text included in the document, and may determine whetherschedule information (e.g., time information or place information) isincluded in the document.

If schedule information is not included in the document, as illustratedin FIG. 8A, the processor 120 determines a scan function as adocument-related function, and provides a second user interfaceincluding a scan button 801 and a graphic effect 803 indicating an areawhere the document is recognized. If schedule information 815 isincluded in the document, as illustrated in FIG. 8B, the processor 120determines a scan function and a schedule add function asdocument-related functions, and may provide a second user interfaceincluding a scan button 811, a schedule add button 813, and a graphiceffect 817 indicating an area where the document is recognized.

As another example, if the document included in the image corresponds tothe second type (e.g., a QR code or a barcode), the processor 120 mayanalyze a code in the document, and may determine whether paymentinformation and web address information are included. If the code in thedocument includes only web address information, as illustrated in FIG.8C, the processor 120 determines a web access function as adocument-related function, and provides a second user interfaceincluding a web access button 831. If the code in the document includesweb address information and payment information, as illustrated in FIG.8D, the processor 120 determines a web access function and a paymentfunction as document-related functions, and provides a second userinterface including a web access button 841 and a payment button 843.

As another example, if the document included in the image corresponds tothe third type (e.g., a book, a research paper, or a leaflet), theprocessor 120 may recognize text included in the document, and maydetermine whether searchable information is included in the document. Ifthe searchable information is included in the document, as illustratedin FIG. 8E, the processor 120 determines a scan function and anadditional information search function as document-related functions,and may provide a second user interface including a scan button 851 andan additional information search button 853. If the searchableinformation is not included in the document, the processor 120 maydetermine a scan function as a document-related function, and mayprovide a second user interface including a scan button.

As another example, if the document included in the image corresponds tothe fourth type (e.g., a business card), the processor 120 may determinea scan function and contact information add button as document-relatedfunctions, and may provide a second user interface including a scanbutton and a contact information add button. The above-described seconduser interfaces may further include at least one first graphic objectrelated to photographing an image and a second graphic object indicatingthe existence of a document in an image.

Referring back to FIG. 7, if the type of document is a type providing asingle function, the electronic device (e.g., the processor 120)determines a single function based on the information included in thedocument as a document-related function, in step 713, and provides asecond user interface including at least one third graphic object forperforming the document-related function determined, in step 711. Forexample, if the document included in the image corresponds to the fifthtype (e.g., miscellaneous documents), the processor 120 may determine ascan function as a document-related function, and may provide a seconduser interface including a scan button.

If the document included in the image is not recognizable, theelectronic device (e.g., the processor 120) provides a third userinterface including an error report message indicating that the documentis not recognizable, in step 715. The error report message may include,for example, a message that requests adjustment of at least one of aphotography location, a photographing direction, or a subject distancefor photography (e.g., the distance between a document and an electronicdevice). The error report message may include, for example, at least oneof information associated with the size of the document, informationrelated to the distortion of the document, information associated with arequest for adjusting the photography location, information associatedwith a request for adjusting the subject distance for photography, orinformation associated with a request for adjusting the photographingdirection. The third user interface may include at least one firstgraphic object related to photographing and an error report message, andmay additionally include at least one of a second graphic objectindicating that a document exists in an image and at least one thirdgraphic object that provides a document-related function. The at leastone third graphic object included in the third user interface may be ina deactivated state. For example, the at least one third graphic objectincluded in the third user interface may be deactivated so as toindicate that the document-related function is not allowed to perform.

FIG. 9 is a flowchart 900 illustrating a process of providing varioususer interfaces based on a predetermined condition, by an electronicdevice, according to an embodiment. Operations of FIG. 9 described belowmay be the detailed operations of at least some of steps 705 to 715 ofFIG. 7. In the embodiment provided hereinafter, steps may be performedsequentially, but it is not necessary. For example, the order of stepsmay be changed, and at least two steps may be performed in parallel.Here, an electronic device may be the electronic device 101 of FIG. 1.Hereinafter, at least some operations of FIG. 9 will be described withreference to FIGS. 10A to 10C, and FIGS. 11A to 11C.

FIG. 10A is a diagram illustrating an example of a user interfaceincluding an error report message related to the size of a document, byan electronic device, according to an embodiment. FIG. 10B is a diagramillustrating an example of a user interface including an error reportmessage related to the size of a document, by an electronic device,according to an embodiment. FIG. 10C is a diagram illustrating anexample of a user interface including an error report message related tothe size of a document, by an electronic device, according to anembodiment. FIG. 11A is a diagram illustrating an example of a userinterface provided based on the distortion level of a document by anelectronic device, according to an embodiment. FIG. 11B is a diagramillustrating an example of a user interface provided based on thedistortion level of a document by an electronic device, according to anembodiment. FIG. 11C is a diagram illustrating an example of a userinterface provided based on the distortion level of a document by anelectronic device, according to an embodiment.

Referring to flowchart 900 of FIG. 9, an electronic device (e.g., theprocessor 120 of FIG. 1) determines whether the size of a document isless than or equal to a predetermined maximum size, in step 901. Forexample, the processor 120 may determine whether the entirety of adocument is included in an image or a part of the document is includedin the image, based on the number of segments or vertices of thedocument, and determines whether the size of the document is less thanor equal to the predetermined maximum size. If all of the four segmentsor the four vertices of the document are detected, the processor 120determines that the entirety of the document is included in the image,and determines that the size of the document is less than or equal tothe predetermined maximum size. If the four segments or four verticesare not detected (e.g., three or fewer segments or three or fewervertices are detected), the processor 120 determines that a part of thedocument is included in the image, and determines that the size of thedocument is greater than the predetermined maximum size.

If the size of the document is greater than the predetermined maximumsize, the electronic device (e.g., the processor 120) provides a thirduser interface including an error report message that requestsdecreasing the size of the document, in step 913. The error reportmessage that requests decreasing the size of the document may be amessage of requesting adjustment of the photography location and/or thesubject distance for photography of a camera so that the entirety of thedocument is included in the image since the size of the document isgreater than the predetermined maximum size.

For example, as illustrated in FIG. 10A, if only a part of a calendar isincluded in the image, the processor 120 provides a third user interfaceincluding an error report message 1001 which requests adjustment toenable the entirety of the calendar to be included in the image and tostore the calendar without distortion. According to an embodiment, thethird user interface may include at least some of graphic elements whichare included in the third user interface, which have been described instep 705 of FIG. 7.

If the size of the document is less than or equal to the predeterminedmaximum size, the electronic device (e.g., the processor 120) determineswhether the size of the document is greater than or equal to apredetermined minimum size, in step 903. According to an embodiment, ifthe entirety of the document is included in the image, the processor 120may measure the size of the entire document, and may compare themeasured size of the entire document and the predetermined minimum size.

If the size of the document is less than the predetermined maximum size,the electronic device (e.g., the processor 120) provides a third userinterface including an error report message that requests increasing thesize of the document, in step 915. The error report message thatrequests increasing the size of the document may be a message thatrequests adjustment of the distance to the document for photography toenable the size of the document to be increased since the size of thedocument is less than the predetermined minimum size.

For example, as illustrated in FIG. 10B, if a small memo, which issmaller than the predetermined minimum size, is included in the image,the processor 120 provides a third user interface including an errorreport message 1011, which requests adjusting the distance to the memofor photography to be closer so that the memory is stored withoutdistortion. As another example, as illustrated in FIG. 10C, if a smallmemo, which is smaller than the predetermined minimum size, is includedin the image, the processor 120 provides a third user interfaceincluding an error report message 1021 and a graphic element 1023. Theerror report message 1021 requests adjusting the distance to the memofor photography to be closer so that the memory is stored withoutdistortion, and the graphic element 1023 indicates a recommended sizeand/or area associated with the memo. The third user interface mayinclude at least some of graphic elements which are included in thethird user interface, which has been described in step 715 of FIG. 7.

If the size of the document is less than or equal to the predeterminedminimum size, the electronic device (e.g., the processor 120) determineswhether a distortion level is greater than a first predetermined level,in step 905. The distortion level may be, for example, a degree ofdistortion of the document included in the image. According to anembodiment, the distortion level may be determined based on the angle ofat least one of the four vertices of the document. The distortion levelmay be determined based on the four segments of the document, or theangle of at least two of the plurality of segments of the document. Thefirst predetermined level may be a level at which a document is scannedand stored without performing an additional process related todistortion (e.g., compensation for distortion), or a level in which textincluded in a document is recognizable without compensation fordistortion. The first predetermined level may be set and/or changed by adesigner and/or a user. For example, the first predetermined level maybe defined to include the case in which the angles of the four verticesof the document are all greater than or equal to a first threshold angle(e.g., 65 degrees). For example, if the angles of the four vertices ofthe document are all greater than or equal to the first threshold angle,the processor 120 may determine that the distortion level of thedocument is less than or equal to the first predetermined level. If theangle of at least one of the four vertices of the document is less thanthe first threshold angle, the processor 120 may determine that thedistortion level of the document is greater than the first predeterminedlevel. This is merely an example provided for ease of understanding, andembodiments are not limited thereto. For example, embodiments maydetermine the distortion level of the document using various methodsknown to those skilled in the art.

If the distortion level of the document is less than or equal to thefirst predetermined level, the electronic device (e.g., the processor120) proceeds with step 707 of FIG. 7 and subsequent operations. Forexample, if the distortion level of the document is less than or equalto the first predetermined level, the processor 120 may perform at leastone of steps 707, 709, 711, or 713 of FIG. 7, may determine adocument-related function based on the type of document, and may providea second user interface including at least one third graphic object forperforming the document-related function. For example, as illustrated inFIG. 11A, if the distortion level of the document is less than or equalto the first predetermined level, the processor 120 may provide a seconduser interface including the scan button 1101.

If the distortion level of the document is greater than the firstpredetermined level, the electronic device (e.g., the processor 120)determines whether the distortion level is less than a secondpredetermined level, in step 907. The second predetermined level may bea level in which a document is scanned and stored after compensation fordistortion of the document, or a level in which text included in thedocument is recognizable after compensation for distortion of thedocument. The second predetermined level may be set and/or changed by adesigner and/or a user. For example, the second predetermined level maybe set to a value greater than the first predetermined level. Forexample, the second predetermined level may be defined to include thecase in which the angles of the four vertices of the document are allgreater than a second threshold value (e.g., 45 degrees), and the angleof at least one of the vertices is less than the first threshold angle.For example, if the angles of the four vertices of the document are allgreater than the second threshold value and the angle of at least onevertex is less than the first predetermined angle, it is determined thatthe distortion level of the document is less than or equal to the secondpredetermined level and greater than the first predetermined level. Ifthe angle of at least one of the four vertices of the document is lessthan the second threshold angle, the processor 120 may determine thatthe distortion level of the document is greater than the secondpredetermined level.

If the distortion level of the document is less than the secondpredetermined level, and is greater than the first predetermined level,the electronic device (e.g., the processor 120) determines adocument-related function based on the type of document, in step 909.The step of determining a document-related function based on the type ofdocument may be the same as steps 707 and 709 of FIG. 7, or may be thesame as steps 707 and 711 of FIG. 7.

The electronic device (e.g., the processor 120) provides a fourth userinterface including at least one third graphic object for performing thedetermined document-related function and a distortion report message, instep 911. The fourth user interface may include at least one graphicelement included in the second user interface, and may additionallyinclude a distortion report message.

For example, as illustrated in FIG. 11B, if the distortion level of thedocument is less than the second predetermined level, and is greaterthan the first predetermined level, the processor 120 provides a fourthuser interface including a scan button 1111 and a distortion reportmessage 1113. The distortion report message may include a messageindicating that a photographing direction and/or the orientation of thedocument needs to be changed in order to clearly scan the documentwithout distortion. According to an embodiment, if a user input to ascan button is detected in the fourth user interface, the processor 120may obtain the document in the image, may compensate for the distortionof the obtained document, and may store the document of which distortionis compensated.

If the distortion level of the document is greater than or equal to thesecond predetermined level, the electronic device (e.g., the processor120) provides a third user interface including a distortion reportmessage, in step 917. The distortion report message may include amessage indicating that a photographing direction and/or the orientationof the document needs to be changed in order to clearly scan thedocument without distortion. According to an embodiment, the third userinterface may include at least some of graphic elements which areincluded in the third user interface, which has been described in step705 of FIG. 7. According to an embodiment, the third user interface mayinclude a deactivated scan button.

For example, as illustrated in FIG. 11C, if the distortion level of thedocument is greater than the second predetermined level, the processor120 provides a third user interface including a deactivated scan button1121 and a distortion report message 1123.

According to various embodiments, a method of an electronic deviceincludes determining whether a document is included in an image providedfrom a camera. When a document is not included in the image, a firstuser interface is provided including at least one first graphic objectfor photographing the image. When a document is included in the image, asecond user interface is provided for providing a function related tothe document. The second user interface includes the at least one firstgraphic object, a second graphic object indicating that a documentexists in the image, and at least one third graphic object forperforming the function related to a document.

The second graphic object may include at least one of a predeterminedsymbol indicating that a document exists in the image or a graphiceffect indicating an area where a document is recognized in the image.

Providing the second user interface that provides the function relatedto the document may include, when a document is included in the image,determining whether the document is recognizable, based on whether thedocument satisfies a predetermined condition. When the document isrecognizable, at least one function related to the document isdetermined based on a type of document or information included in thedocument.

The at least one function related to the document may include at leastone of a function of scanning the document, a function of searching foradditional information related to the document, a payment function basedon information included in the document, a web access function based oninformation included in the document, a schedule add function based oninformation included in the document, and a function of adding contactinformation based on information included in the document.

The second user interface may include a third graphic object thatprovides a function of scanning the document, and the method may furtherinclude, when a user input to the third graphic object, which providesthe function of scanning the document, is detected in the state in whichthe second user interface is provided, extracting and storing only thedocument from the image. When a user input to the first graphic objectis detected in the state in which the second user interface is provided,the image including the document is photographed and stored.

Determining whether the document is recognizable may include determiningwhether at least one of a size of the document or a distortion level ofthe document satisfies the predetermined condition.

The method may further include providing, by the processor, a third userinterface including an error report message related to at least one of asize of the document or a distortion level of the document, when thedocument is not recognizable. The third user interface may furtherinclude at least one of the at least one first graphic object and thesecond graphic object indicating that a document exists in the image.

The third user interface may further include at least one third graphicobject for performing a function related to the document, and the atleast one third graphic object is in a deactivated state.

The electronic device according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may include,for example, a portable communication device (e.g., a smartphone), acomputer device, a portable multimedia device, a portable medicaldevice, a camera, a wearable device, or a home appliance. According toan embodiment of the disclosure, the electronic devices are not limitedto those described above.

It should be appreciated that various embodiments of the presentdisclosure and the terms used therein are not intended to limit thetechnological features set forth herein to particular embodiments andinclude various changes, equivalents, or replacements for acorresponding embodiment. With regard to the description of thedrawings, similar reference numerals may be used to refer to similar orrelated elements. It is to be understood that a singular form of a nouncorresponding to an item may include one or more of the things, unlessthe relevant context clearly indicates otherwise. As used herein, eachof such phrases as “A or B,” “at least one of A and B,” “at least one ofA or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least oneof A, B, or C,” may include any one of, or all possible combinations ofthe items enumerated together in a corresponding one of the phrases. Asused herein, such terms as “1st” and “2nd,” or “first” and “second” maybe used to simply distinguish a corresponding component from another,and does not limit the components in other aspect (e.g., importance ororder). It is to be understood that if an element (e.g., a firstelement) is referred to, with or without the term “operatively” or“communicatively”, as “coupled with,” “coupled to,” “connected with,” or“connected to” another element (e.g., a second element), it means thatthe element may be coupled with the other element directly (e.g.,wiredly), wirelessly, or via a third element.

As used herein, the term “module” may include a unit implemented inhardware, software, or firmware, and may interchangeably be used withother terms, for example, “logic,” “logic block,” “part,” or“circuitry”. A module may be a single integral component, or a minimumunit or part thereof, adapted to perform one or more functions. Forexample, according to an embodiment, a module may be implemented in aform of an application-specific integrated circuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program 140) including one or more instructions that arestored in a storage medium (e.g., internal memory 136 or external memory138) that is readable by a machine (e.g., the electronic device 101).For example, a processor (e.g., the processor 120) of the machine (e.g.,the electronic device 101) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it, with orwithout using one or more other components under the control of theprocessor. This allows the machine to be operated to perform at leastone function according to the at least one instruction invoked. The oneor more instructions may include a code generated by a complier or acode executable by an interpreter. The machine-readable storage mediummay be provided in the form of a non-transitory storage medium. Wherein,the term “non-transitory” simply means that the storage medium is atangible device, and does not include a signal (e.g., an electromagneticwave), but this term does not differentiate between where data issemi-permanently stored in the storage medium and where the data istemporarily stored in the storage medium.

According to an embodiment, a method may be included and provided in acomputer program product. The computer program product may be traded asa product between a seller and a buyer. The computer program product maybe distributed in the form of a machine-readable storage medium (e.g.,compact disc read only memory (CD-ROM)), or be distributed (e.g.,downloaded or uploaded) online via an application store (e.g.,PlayStore™), or between two user devices (e.g., smart phones) directly.If distributed online, at least part of the computer program product maybe temporarily generated or at least temporarily stored in themachine-readable storage medium, such as memory of the manufacturer'sserver, a server of the application store, or a relay server.

According to various embodiments, each component (e.g., a module or aprogram) of the above-described components may include a single entityor multiple entities. According to various embodiments, one or more ofthe above-described components may be omitted, or one or more othercomponents may be added. Alternatively or additionally, a plurality ofcomponents (e.g., modules or programs) may be integrated into a singlecomponent. In such a case, according to various embodiments, theintegrated component may still perform one or more functions of each ofthe plurality of components in the same or similar manner as they areperformed by a corresponding one of the plurality of components beforethe integration. According to various embodiments, operations performedby the module, the program, or another component may be carried outsequentially, in parallel, repeatedly, or heuristically, or one or moreof the operations may be executed in a different order or omitted, orone or more other operations may be added.

While the disclosure has been particularly shown and described withreference to certain embodiments thereof, it will be understood by thoseof ordinary skill in the art that various changes in form and detailsmay be made therein without departing from the spirit and scope of thedisclosure as defined by the appended claims and their equivalents.

What is claimed is:
 1. An electronic device, comprising: a camera: adisplay; and a processor, wherein the processor is configured to:determine whether a document is included in an image provided from thecamera; when the document is not included in the image, provide a firstuser interface to the display that includes the image and at least onefirst graphic object for photographing the image; and when the documentis included in the image, provide a second user interface to the displayfor providing a function related to the document, and wherein the seconduser interface comprises the image, the at least one first graphicobject, a second graphic object indicating that the document exists inthe image, and at least one third graphic object for performing afunction related to the document.
 2. The electronic device of claim 1,wherein the second graphic object comprises at least one of apredetermined symbol indicating that the document exists in the image,and a graphic effect indicating an area where the document is recognizedin the image.
 3. The electronic device of claim 1, wherein: when thedocument is included in the image, the processor is further configuredto determine whether the document is recognizable, based on whether thedocument satisfies a predetermined condition; and when the document isrecognizable, the processor is further configured to determine at leastone function related to the document, based on a type of document orinformation included in the document.
 4. The electronic device of claim3, wherein the at least one function related to the document comprisesat least one of a function of scanning the document, a function ofsearching for additional information related to the document, a paymentfunction based on information included in the document, a web accessfunction based on information included in the document, a schedule addfunction based on information included in the document, and a functionof adding contact information based on information included in thedocument.
 5. The electronic device of claim 4, wherein: the second userinterface comprises a third graphic object that provides a function ofscanning the document; when a user input to the third graphic object isdetected in a state in which the second user interface is provided, theprocessor is further configured to extract and store only the documentfrom the image; and when a user input to the first graphic object isdetected in a state in which the second user interface is provided, theprocessor is further configured to photograph the image including thedocument and store the image.
 6. The electronic device of claim 3,wherein the processor is further configured to determine whether atleast one of a size of the document or a distortion level of thedocument satisfies the predetermined condition.
 7. The electronic deviceof claim 6, wherein: when the document is not recognizable, theprocessor is further configured to provide a third user interfaceincluding an error report message related to at least one of the size ofthe document and the distortion level of the document; and the thirduser interface further comprises at least one of the at least one firstgraphic object and the second graphic object indicating that thedocument exists in the image.
 8. The electronic device of claim 7,wherein: the third user interface further comprises the at least onethird graphic object for performing a function related to the document;and the at least one third graphic object is in a deactivated state. 9.An operation method of an electronic device, comprising: determiningwhether a document is included in an image provided from a camera of theelectronic device; when the document is not included in the image,providing a first user interface to a display of the electronic devicethat includes the image and at least one first graphic object forphotographing the image; and when the document is included in the image,providing a second user interface to the display for providing afunction related to the document, wherein the second user interfacecomprises the image, the at least one first graphic object, a secondgraphic object indicating that the document exists in the image, and atleast one third graphic object for performing a function related to thedocument.
 10. The method of claim 9, wherein the second graphic objectcomprises at least one of a predetermined symbol indicating that thedocument exists in the image, and a graphic effect indicating an areawhere the document is recognized in the image.
 11. The method of claim9, wherein providing the second user interface comprises: when thedocument is included in the image, determining whether the document isrecognizable, based on whether the document satisfies a predeterminedcondition; and when the document is recognizable, determining at leastone function related to the document, based on a type of document orinformation included in the document.
 12. The method of claim 11,wherein the at least one function related to the document comprises atleast one of a function of scanning the document, a function ofsearching for additional information related to the document, a paymentfunction based on information included in the document, a web accessfunction based on information included in the document, a schedule addfunction based on information included in the document, and a functionof adding contact information based on information included in thedocument.
 13. The method of claim 12, wherein: the second user interfacecomprises a third graphic object that provides a function of scanningthe document; and the method further comprises: when a user input to thethird graphic object is detected in a state in which the second userinterface is provided, extracting and storing only the document from theimage; and when a user input to the first graphic object is detected ina state in which the second user interface is provided, photographingand storing the image including the document.
 14. The method of claim11, wherein determining whether the document is recognizable comprises:determining whether at least one of a size of the document or adistortion level of the document satisfies the predetermined condition.15. The method of claim 12, further comprising: providing, by theprocessor, a third user interface including an error report messagerelated to at least one of a size of the document and a distortion levelof the document, when the document is not recognizable, wherein thethird user interface further comprises at least one of the at least onefirst graphic object and the second graphic object indicating that thedocument exists in the image.
 16. The method of claim 15, wherein thethird user interface further comprises the at least one third graphicobject for performing a function related to the document, and the atleast one third graphic object is in a deactivated state.
 17. Acomputer-readable recording medium storing a program to implementoperations, the operations comprising: determining whether a document isincluded in an image provided from a camera; when the document is notincluded in the image, providing a first user interface to a displaythat includes the image and at least one first graphic object forphotographing the image; and when the document is included in the image,providing a second user interface to the display that provides afunction related to the document, wherein the second user interfacecomprises the image, the at least one first graphic object, a secondgraphic object indicating that the document exists in the image, and atleast one third graphic object for performing a function related to thedocument.
 18. The computer-readable recording medium of claim 17,wherein the second graphic object comprises at least one of apredetermined symbol indicating that the document exists in the image,and a graphic effect indicating an area where the document is recognizedin the image.
 19. The computer-readable recording medium of claim 17,wherein providing the second user interface comprises: when the documentis included in the image, determining whether the document isrecognizable, based on whether at least one of a size of the document ora distortion level of the document satisfies a predetermined condition;and when the document is recognizable, determining at least one functionrelated to the document, based on a type of document or informationincluded in the document.
 20. The computer-readable recording medium ofclaim 17, wherein: the second user interface comprises a third graphicobject for providing a function of scanning the document; and thecomputer-readable recording medium stores a program for furtherimplementing: when a user input to the third graphic object is detectedin a state in which the second user interface is provided, extractingand storing the only document from the image; and when a user input tothe first graphic object is detected in a state in which the second userinterface is provided, photographing and storing the image including thedocument.